PASSION: Parallel And Scalable Software for Input-Output

We are developing a software system called PASSION: Parallel And Scalable Software for Input-Output which provides software support for high performance parallel I/O. PASSION provides support at the language, compiler, runtime as well as file system level. PASSION provides runtime procedures for parallel access to files (read/write), as well as for out-of-core computations. These routines can either be used together with a compiler to translate out-of-core data parallel programs written in a language like HPF, or used directly by application programmers. A number of optimizations such as Two-Phase Access, Data Sieving, Data Prefetching and Data Reuse have been incorporated in the PASSION Runtime Library for improved performance. PASSION also provides an initial framework for runtime support for out-of-core irregular problems. The goal of the PASSION compiler is to automatically translate out- of-core data parallel programs to node programs for distributed memory machines, with calls to the PASSION Runtime Library. At the language level, PASSION suggests extensions to HPF for out-of-core programs. At the file system level, PASSION provides support for buffering and prefetching data from disks. A portable parallel file system is also being developed as part of this project, which can be used across homogeneous or heterogeneous networks of workstations. PASSION also provides support for integrating data and task parallelism using parallel I/O techniques. We have used PASSION to implement a number of out-of-core applications such as a Laplace\u27s equation solver, 2D FFT, Matrix Multiplication, LU Decomposition, image processing applications as well as unstructured mesh kernels in molecular dynamics and computational fluid dynamics. We are currently in the process of using PASSION in applications in CFD (3D turbulent flows), molecular structure calculations, seismic computations, and earth and space science applications such as Four-Dimensional Data Assimilation. PASSION is currently available on the Intel Paragon, Touchstone Delta and iPSC/860. Efforts are underway to port it to the IBM SP-1 and SP-2 using the Vesta Parallel File System

ARTICLE NO. PC971367 A Library-Based Approach to Task Parallelism in a Data-Parallel Language

Pure data-parallel languages such as High Performance Fortran version 1 (HPF) do not allow efficient expression of mixed task/data-parallel computations or the coupling of separately compiled data-parallel modules. In this paper, we show how these common parallel program structures can be represented, with only minor extensions to the HPF model, by using a coordination library based on the Message Passing Interface (MPI). This library allows data-parallel tasks to exchange distributed data structures using calls to simple communication functions. We present microbenchmark results that characterize the performance of this library and that quantify the impact of optimizations that allow reuse of communication schedules in common situations. In addition, results from two-dimensional FFT, convolution, and multiblock programs demonstrate that the HPF/ MPI library can provide performance superior to that of pure HPF. We conclude that this synergistic combination of two parallel programming standards represents a useful approach to task parallelism in a data-parallel framework, increasing the range of problems addressable in HPF without requiring complex compile

Efficient collective data transfer mechanisms for high-performance parallel and distributed computing environments

Advances in computing and networking infrastructure have enabled an increasing number of applications to utilize resources that are geographically distributed. High-performance distributed computing environments present significant new problems and opportunities for programming language designers, compiler implementors, and runtime system developers. This thesis addresses several issues in providing efficient mechanisms for collective data transfer among sets of processes in distributed environments. In particular, it examines how such mechanisms are employed in two contexts to achieve high performance in application programs. The techniques are designed to provide efficiency, portability, and ease of programming. Many scientific applications need to access data resources from multiple locations in a distributed computing environment. This thesis presents the design and implementation of a system called RIO that provides remote I/O functionality. Programs use familiar parallel I/O interfaces to access remote file systems. The RIO library offers significant performance, flexibility, and convenience advantages over existing techniques for remote data access such as manual staging and deployment of distributed file systems. Mechanisms provided by the Globus Metacomputing Toolkit are used to support the integration of the library in a distributed environment. Experiments using synthetic benchmarks and real applications that use RIO demonstrate performance benefits over staging techniques and current distributed file systems. Programming models for distributed systems will need to exploit all types of parallelism within application domains. By utilizing a mixture of both task and data parallelism in parallel applications, one may extend the range of problems that can be solved efficiently beyond what is possible with pure data-parallel programming languages (such as High Performance Fortran) alone. This thesis proposes a programming model where common parallel program structures can be represented, with only minor extensions to the HPF model, by using a coordination library based on the Message Passing Interface (MPI). This library allows data-parallel tasks to exchange distributed data structures using calls to simple communication functions. Experiments with microbenchmarks and applications characterize the performance of this library and quantify the impact of various optimizations. These results demonstrate that the HPF/MPI library can provide performance superior to that of pure HPF

MPI as a Coordination Layer for Communicating HPF Tasks

Data-parallel languages such as High Performance Fortran (HPF) present a simple execution model in which a single thread of control performs high-level operations on distributed arrays. These languages can greatly ease the development of parallel programs. Yet there are large classes of applications for which a mixture of task and data parallelism is most appropriate. Such applications can be structured as collections of data-parallel tasks that communicate by using explicit message passing. Because the Message Passing Interface (MPI) defines standardized, familiar mechanisms for this communication model, we propose that HPF tasks communicate by making calls to a coordination library that provides an HPF binding for MPI. The semantics of a communication interface for sequential languages can be ambiguous when the interface is invoked from a parallel language; we show how these ambiguities can be resolved by describing one possible HPF binding for MPI. We then present the design of a li..

Integrating Task and Data Parallelism Using Parallel I/O Techniques

Task and data parallelism are traditionally seen as disjoint modes of parallelism. But a lot of multidisciplinary applications can benefit from both types of parallelism. The development of an integrated task/data parallel programming system introduces challenging technical issues in the areas of language design, compilation, and runtime system design. This paper concentrates on communication issues in an integrated task/data parallel system. The paper first presents a semantic model for communicationbetween two data-parallel tasks. Various implementation approaches to realize these semantics are then discussed. We evaluate these different approaches using an image processing example as a case study. 1 Introduction In data-parallel programming, programs apply a sequence of similar operations to all or most elements of a large data structure; in task-parallel programming, programs consist of a set of (potentially dissimilar) parallel tasks that perform explicit communication and synch..

Remote I/O: Fast Access to Distant Storage

As high-speed networks make it easier to use distributed resources, it becomes increasingly common that applications and their data are not colocated. Users have traditionally addressed this problem by manually staging data to and from remote computers. We argue instead for a new remote I/O paradigm in which programs use familiar parallel I/O interfaces to access remote filesystems. In addition to simplifying remote execution, remote I/O can improve performance relative to staging by overlapping computation and data transfer or by reducing communication requirements. However, remote I/O also introduces new technical challenges in the areas of portability, performance, and integration with distributed computing systems. We propose techniques designed to address these challenges and describe a remote I/O library called RIO that we have developed to evaluate the effectiveness of these techniques. RIO addresses issues of portability by adopting the quasi-standard MPI-IO interface and by de..

Double Standards: Bringing Task Parallelism to HPF via the Message Passing Interface

High Performance Fortran (HPF) does not allow efficient expression of mixed task/dataparallel computations or the coupling of separately compiled data-parallel modules. In this paper, we show how a coordination library implementing the Message Passing Interface (MPI) can be used to represent these common parallel program structures. This library allows data-parallel tasks to exchange distributed data structures using calls to simple communication functions. We present microbenchmark results that characterize the performance of this library and that quantify the impact of optimizations that allow reuse of communication schedules in common situations. In addition, results from two-dimensional FFT, convolution, and multiblock programs demonstrate that the HPF/MPI library can provide performance superior to that of pure HPF. We conclude that this synergistic combination of two parallel programming standards represents a useful approach to task parallelism in a data-parallel framework, incre..